Application of majority voting to pattern recognition: an analysis of its behavior and performance

Recently, it has been demonstrated that combining the decisions of several classifiers can lead to better recognition results. The combination can be implemented using a variety of strategies, among which majority vote is by far the simplest, and yet it has been found to be just as effective as more complicated schemes in improving the recognition results. However, all the results reported thus far on combinations of classifiers have been experimental in nature. The intention of this research is to examine the mode of operation of the majority vote method in order to gain a deeper understanding of how and why it works, so that a more solid basis can be provided for its future applications to different data and/or domains. In the course of our research, we have analyzed this method from its foundations and obtained many new and original results regarding its behavior. Particular attention has been directed toward the changes in the correct and error rates when classifiers are added, and conditions are derived under which their addition/elimination would be valid for the specific objectives of the application. At the same time, our theoretical findings are compared against experimental results, and these results do reflect the trends predicted by the theoretical considerations.